Acquiring  Evolving 
Technologies: 

Web  Services  Standards 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

26  OCT  2006  2  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2006  to  00-00-2006 

4.  TITLE  AND  SUBTITLE 

Acquiring  Evolving  Technologies:  Web  Services  Standards 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Carnegie  Mellon  University  , Software  Engineering  Institute 
(SEI), Pittsburgh, PA, 15213 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  OS 

unclassified  unclassified  unclassified  Report  (SAR) 

22 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Acquiring  Evolving  Technologies 


Purpose:  combine  ideas  from  different  systems  engineering  areas  into  a 
repeatable  process  for  managing  technology  assessments 

This  presentation  discusses 

•  challenges  of  acquiring  Web  services 

•  why  assess  technology? 

•  assessing  technology  appropriateness 

•  applicability  to  net-centricity 

Although  not  detailed,  this  presentation  borrows  from 

•  system  and  software  architecture 

•  business  principles 

•  process  improvement 

•  technology  solutions 

•  system  of  systems  techniques 
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Symbols  Used  in  This  Presentation 


Concept  ? 


Example  ? 


Technology  ? 


=  Software  Engineering  Institute  CarnegieMellon 


Acquiring  Evolving  Technologies: 
Web  Services  Standards 


©2006  Carnegie  Mellon  University 


Acquisition  Challenges 


Architectural  and 
Design  Decisions 
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First,  a  Notional  Program 


A  notional  program,  Language  Translation  Services  (LTS),  helps  us 
explore  this  topic  within  a  specific  context. 


LTS  Version  1  (2005) 

•  Purpose:  translate  a  paragraph  of  text  from  one  language  to 
another 

Features 

•  anyone  in  the  world  can  create  and/or  use  a  translation  service 

•  customization  of  features  (such  as  accuracy,  speed,  and  dialect) 
is  supported 
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LTS  Upgrade 


LTS  Version  2  (2006) 

•  Goal;  improve  accuracy 


New  Features 

•  Link  up  to  10  paragraphs;  changes  to  previous  translation 
responses  may  be  returned 

•  request  translations  with  additional  features  including  domain, 
linking,  and  alternate  choices  when  the  accuracy  of  translation  is 
less  than  98% 

•  the  service  must  report  state  changes  within  10  seconds  (for 
example,  degraded  performance) 
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LTS  Architectural  Solution 


A  service-oriented  architecture  (SOA)  was  selected  as  the 
architecture  for  LTS  Version  1. 

SOAs  have  been  described  as 

•  “SOA  is  about  separation”  — CBDI 

•  “supports  integrating  your  business  as  linked,  repeatable,  business 
tasks” 

—IBM 

•  “a  lifestyle”  and  “something  you  do,  not  something  you  buy”  — Burton 
Group 

Issues  with  SOAs  that  we  will  not  discuss  today 

•  organizational  and  cultural  change 

•  governance 

•  infrastructure 

•  adoption  techniques 

•  implementation  techniques 
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SOA  and  Quality  Attributes 


Using  an  SOA  approach  impacts  the  quality  attributes  in 
different  ways. 


Positive  Impact 

Interoperability 

Extensibility 

Adaptability 

Modifiability 


Neutral  Impact 

Reliability 

Availability 

Scalability 

Usability 

Operability  and 
Deployability 


Negative  Impact 

Security 

Performance 

Testability 

Auditability 


[O’Brien  05]  Quality  Attributes  and  Service-Oriented  Architectures  (CMU/SEI-2005-TN-014) 
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Implementing  an  SOA  Using 
Web  Services  Standards 

Think  of  Web  services  standards  (WS-*)  as  a  tool  for  SOA  technology 
(Burton  Group)  or  standards-based  SOA  (Sonic). 

Launched  in  the  year  2000,  arguably 

•  six  years  old;  today’s  hot  topic 

—  adolescent  or  mature? 

From  50  to  240  specifications 

•  open  framework  with  a  large  number  of  commercial  solutions 

—  options  or  confusion? 

Three  organizations  manage  the  open  standards 

•  many  companies  large  and  small  participating 

—  cooperating  or  competing? 
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Why  Should  We  Assess  Technologies? 


Risks  related  to  acquiring  technology 

•  complexity  of  implementation 

•  testing  challenges 

•  managing  change 

—  neither  technology  nor  programs  stand  still 

DoD  policy  requires  for  Major  Defense  Acquisition  Programs  (MDAPs) 
and  Major  Acquisition  Information  Systems  (MAIS)  programs 

•  Technology  Readiness  Assessment  (TRA)  per  DoD  5000.2  usually 
via  Technology  Readiness  Levels  (TRLs) 

•  TRLs  assign  a  single  number,  which  especially  for  software,  does 
not  address  the  many  dimensions  of  readiness  assessment. 
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Beyond  Technology  Readiness  Levels 


Simple,  yet  meaningful  method  to  assessment 

•  prototypes  or  models  are  meaningful,  but  difficult  and  time-consuming  to  create 

•  white  paper  research  is  not  deep  enough 

•  Is  there  something  in  between? 

Change:  a  key  challenge  of  assessment 

•  wait  until  stable  >  nothing  gets  done 

•  blindly  go  ahead  >  everything  gets  confused 

•  keep  changing  the  decision  >  everyone  gets  confused 

Dimensions  of  the  assessment 

•  ability  to  meet  the  requirements 

•  environmental  appropriateness  and  constraints 

•  importance  to  the  solution 

•  lifecycle  match  [Smith  04] 

Processes  within  the  acquisition  life  cycle  must  allow  decisions  to  be 
reevaluated  on  a  regular  basis. 


[Smith  04]  An  Alternative  to  Technology  Readiness  Levels  for  Non-Developmental  Item  (NDI)  Software  (CMU/SEI-2004-TR-01 3) 
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Assessing  Web  Services  Standards 


WS-*  standards 

•  How  effective  is  each  standard? 

•  Where  is  each  standard  in  the  process? 

•  How  much  effort  is  being  put  into  developing  the  standard? 

•  conflicting  and/or  competing  standards? 

•  compatibility  and  certification? 

Standards  process,  W3C,  OASIS,  WS-I 

•  Which  companies  are  participating? 

•  What  impact  are  they  having  on  the  process? 

Products  available 

•  companies  implementing  and  advertising  WS-*? 

•  tools  to  develop  and  manage  WS-*  solutions? 

•  market  acceptance,  availability? 

•  opinions  of  external  research  organizations? 
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Assessment  Dimensions 


Assessing  a  standard’s  maturity 

•  rate  of  change 

•  number  of  features 

•  number  of  features  not  available 

•  number  of  implementations  available 
Assessing  a  standard’s  impact 

•  enable,  inhibit,  or  add  confusion  to  system  implementation 

•  trade-off  decisions  to  be  made 

•  potential  changes  to  standards,  how  it  affects  architectural  decisions 
Proposed  Analysis  Method 

•  compare  the  needed  system  capabilities  to  SOA  quality  attributes 

•  match  them  with  the  appropriate  Web  service  standards  and 

•  assess  the  WS-*  maturity  and  impact  on  the  system 
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Initial  Analysis  of  LTS  Version  1 


LTS  Capabilities 


Add  New  Services 


World-Wide, 
Multiple  Services 


Assorted 

Functionality 


World-Wide, 
Multiple  Users 


SOA  Quality 
Attributes 

(SEI  Technical 
Note) 


lnteroperability(+) 

Availability(.) 


Web  Services 
Maturity  and 
Impact 

(Authors’ 

Analysis) 

WS-Discovery(-) 

WS-BPEL(.) 


Extensibility^) 

Scalability(.) 

Performance(-) 


Adaptability(+) 

Modifiability(+) 


WSDL(+) 
ASAP(-) 
nsferf 


WS-Coordinatior 

^^WS-Context(-) 


lnteroperab7lity(+) 

Availability(.) 


WS-Trust(-) 

UDDI(+) 
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Improved  Analysis  for  LTS  Version  1 


WS Standard:  Web  Services  Security  (WS-Security) 
Organization:  OASIS,  Ver:  1.0  3/04 
Impact 


Adaptability 

Auditability 


Modifiability 


Operability  and 
Deployability 

Performance 


Maturity 


Minimal 


Not  key  QA 


More  information  needs  to  be  audited 


'"Availability 

Extensibility 

Interoperability  Positive 


Establish  secure  communication  but  no 
guarantee  of  service  failure 


Security  messages  are  extensible  and 
additional  fields  can  be  added 


Allows  for  loose  or  tightly  coupled 
systems,  requires  policies  to  be  well 
defined 


Underlying  service  can  change  without 
change  in  message 


Minimal 


Not  key  QA 

Additional  message  and  increased  size 


Establish  secure  communication 


Built  for  confidential  message 
transmission 


More  messages  and  scenarios  to  be 
tested 


Although  widely  implemented,  this  key  QA 
may  be  affected 


As  testing  is  addressed  better,  changes 
might  happen 


Mature 


Impact  A  verage:  0.15 


Widely  implemented 

Maturity  A  verage:  0.69 
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Comparison  of  Select  Standards 


Standard 

Impact 
(2QQ5) . 

Impact 

(2006) 

Maturity 

(2005) 

Maturity 

(2006) 

SOAP 

0.15 

0.15 

0.77 

0.85 

WSDL 

0.23 

0.38 

0.69 

0.31 

UDDI 

0.38 

0.38 

0.62 

0.62 

WS-Security 

0.15 

0.15 

0.69 

0.54 

WS-BPEL 

0.08 

0.23 

-0.31 

-0.62 

WS-Transfer 

0.00 

0.00 

-0.15 

0.08 

WS-Trust 

0.00 

0.00 

-0.54 

-0.54 

WS-Coordination 

0.23 

0.23 

0.69 

-0.54 

WS-Context 

0.15 

0.31 

-1.00 

-0.15 

WS-Discovery 

0.15 

0.15 

-1.00 

-1.00 

-10  1  -10  1 

Negative  Minimal  Positive  Immature  Adolescent  Mature 

◄ - ►  ◄- - ► 
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Net-Centric  Acquisition  Challenges 


Operational 

•  implement  capability  using  varied  and  distributed  systems 

Interoperable 

•  address  system-of-system  issues,  such  as  emergent  properties 

Evolution 

•  handle  changes  in  technology  while  keeping  the  program 
operational  and  interoperable 


SOAs  and  Web  services  standards  are  a  natural  fit  for  net-centric 
solutions  because  of  their  positive  quality  attributes.  However,  they 
bring  with  them  negative  attributes  that  complicate  implementation. 
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LTS  Assessment,  Including  Net-Centric  Objectives.) 


LTS  Capabilities 

Version 

SOA  Quality 
Attributes 

(SEI  Technical 
Note) 

Web  Services 
Maturity  and  Impact 

(Authors’  Analysis) 

NESI  Enterprise 
Technology 
Objectives 

Add  New  Services 

Version  1 

lnteroperability(+) 

Availability(.) 

WS-Discovery(-) 

WS-BPEL(.) 

UDDI(+) 

Capability  Oh 
Demand 

World-Wide,  Multiple 
Services 

Version  1 

Extehsibifity(+) 

Scalability(.) 

Performance(-) 

. WSDL(+) . 

ASAP(-) 

WS-Transfer(.) 

Distributed 

Operations 

Assorted 

Functionality 

Version  1 

Adaptabiiity(+) 

Modifiability(+) 

WS-C66rdiriati6n(-) 

WS-Context(-) 

Customized 

Applications 

World-Wide,  Multiple 
Users 

Version  1 

lnteroperability(+) 

Availabiiity(.) 

. WS-Trust(-) . 

UDDI(+) 

Multi-user  Access 
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LTS  Assessment,  Including  Net-Centric  Objectives2 


LTS  Capabilities 

Version 

SOA  Quality 
Attributes 

(SEI  Technical 
Note) 

Web  Services 
Maturity  and  Impact 

(Authors’  Analysis) 

NESI  Enterprise 
Technology 
Objectives 

Linking  and  Dialects 

Version  2 

Adaptability(+) 

Operability  and 
Deployability(.) 

WS-BPEL(.) 

WS-Policy(-) 

Customized  Delivery 

Auditing  arid  Security 

Future 

Auditability(-) 

Reliability(.) 

Security(-) 

WS-Poiicy(-) 

WS-Security(-) 

WS-Trust(-) 

Assured  Sharing 

New  Features 

Version  2 

Testability(-) 

Extensibility^) 

WS-Policy(-) 

WS-BPEL(.) 

UDDI(+) 

Incremental  Upgrade 

Share  Translations 

Future 

Usability(-) 

Performance(-) 

. SOAP(+) . 

WS-Reliability(-) 

Data  Exchange 
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Summary 


We  need  a  method  to  systematically  assess  the  appropriateness  of 
evolving  technologies. 

•  Technologies  change  frequently,  therefore  the  decisions  based  on 
technology  should  be  reviewed  regularly. 

Quality  attributes  constitute  a  key  dimension  of  technology 
assessments. 

•  For  the  LTS  example,  we  assessed  the  impact  and  maturity 
dimensions. 

Assess  Web  services  standards  regularly  to  reduce  risk. 

•  Apply  this  assessment  tool  and  the  associated  process  to  start, 
then  tailor  each  to  meet  programs’  needs. 
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For  More  Information 


Acquiring  Evolving  Technologies: 

Web  Services  Standards 

Harry  L.  Levinson 
Liam  O'Brien 

Technical  Note 
CMU/SEI-2006-TN-001 

http://www.sei.cmu.edu/publications/documents/06.reports/06tn001.html 
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Contact  Information 


Harry  L.  Levinson 

Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 
412-268-4148 
hll@sei.cmu.edu 


Acquisition  Support  Program 

http://www.sei.cmu.edu/programs/acquisition-support/ 


=  Software  Engineering  Institute  CarnegieMellon 


Acquiring  Evolving  Technologies: 
Web  Services  Standards 


22 


©2006  Carnegie  Mellon  University 


